Analysis Of Machine Learning Techniques Research Articles

A comparative analysis of machine learning techniques for imbalanced data

A comparative analysis of machine learning techniques for imbalanced data

Comparative analysis of machine learning techniques for predicting drilling rate of penetration (ROP) in geothermal wells: A case study of FORGE site

Enhanced Geothermal Systems (EGS) present a compelling means to unlock the considerable yet largely untapped thermal energy within the earth's crust globally. The EGS drilling cost constitutes a substantial portion, up to 60 %, of the overall expenses. Consequently, streamlining and optimizing the drilling processes for these systems hold immense economic significance, with the potential to substantially lower costs, advance the utilization of geothermal energy, and contribute significantly to the reduction of carbon emissions. This research aims to enhance EGS drilling operations by seeking ways of reducing drilling cost.In this research, we harnessed the predictive power of 10 state-of-the-art machine learning (ML) algorithms to anticipate a crucial drilling parameter: ROP. Using the FORGE dataset, we developed a code tailored for the intensive preprocessing of drilling data. This code offers many options, including various noise-removing techniques and scaling approaches. However, the primary focus of our work extends to quantifying uncertainties intrinsic to the predictions of the 10 algorithms employed. To achieve this, a comprehensive approach involved subjecting each algorithm to 40 runs, utilizing the best model from the tuning process.The results show that unaddressed uncertainties may lead to unstable model behavior, where small changes in the input data or random initialization result in significant prediction variations. Thus, models that do not account for uncertainty may be overfit to the noise in the training data, leading to poor generalization to new, unseen data. Remarkably, the results highlight the superiority of the Extreme Tree, Light Gradient Boost, Random Forest, and Gradient Boosting algorithms, showcasing mean absolute error (MAE) and R2 values within the ranges of 3.5–4.5 and 0.9–0.95, respectively. Conversely, artificial neural networks and support vector machine algorithms demonstrate comparatively lower performance, with MAE ranging from 4.7 to 6.3 and R2 from 0.85 to 0.91.Although the results presented in this work are only based on the drilling parameters from one well of the FORGE site and do not include rock properties or geological parameters, however, the proposed nuanced understanding of algorithmic performance is valuable for refining predictive models in geothermal drilling applications, ensuring robustness and reliability in the face of diverse operational scenarios. Navigating a successful workflow, adeptly processing data, and meticulously quantifying uncertainties linked to each predictive algorithm emerge as formidable yet indispensable tasks. Our research not only sheds light on these complexities but also paves the way for a strategic optimization approach based on data-driven ROP models. Therefore, the potential for substantial savings in real-time drilling operations becomes apparent, unlocking a new realm of possibilities for the geothermal energy sector.

Comparative Analysis of Machine Learning Techniques for Non-Intrusive Load Monitoring

Non-intrusive load monitoring (NILM) has emerged as a pivotal technology in energy management applications by enabling precise monitoring of individual appliance energy consumption without the requirements of intrusive sensors or smart meters. In this technique, the load disaggregation for the individual device is accrued by the recognition of their current signals by employing machine learning (ML) methods. This research paper conducts a comprehensive comparative analysis of various ML techniques applied to NILM, aiming to identify the most effective methodologies for accurate load disaggregation. The study employs a diverse dataset comprising high-resolution electricity consumption data collected from an Estonian household. The ML algorithms, including deep neural networks based on long short-term memory networks (LSTM), extreme gradient boost (XgBoost), logistic regression (LR), and dynamic time warping with K-nearest neighbor (DTW-KNN) are implemented and evaluated for their performance in load disaggregation. Key evaluation metrics such as accuracy, precision, recall, and F1 score are utilized to assess the effectiveness of each technique in capturing the nuanced energy consumption patterns of diverse appliances. Results indicate that the XgBoost-based model demonstrates superior performance in accurately identifying and disaggregating individual loads from aggregated energy consumption data. Insights derived from this research contribute to the optimization of NILM techniques for real-world applications, facilitating enhanced energy efficiency and informed decision-making in smart grid environments.

An advanced framework for net electricity consumption prediction: Incorporating novel machine learning models and optimization algorithms

In recent years, the escalating demand for electric energy has underscored the need for robust prediction models capable of accurately anticipating consumption patterns. The imperative lies in enabling utilities and policymakers to optimize resource allocation, strategically plan infrastructure development, and ensure the stability and efficiency of the power grid. This study undertakes a comprehensive comparative analysis of machine learning techniques employed in predicting net electricity consumption in Turkey. The primary goal is to augment the accuracy and performance of electricity load forecasting, thereby contributing to effective energy management and fostering sustainable development within the power sector. Two machine learning models, including CatBoost and Extreme Gradient Boosting (XGBoost), are strategically integrated with optimization algorithms such as Sparrow Search Algorithm (SSA), Phasor Particle Swarm Optimization (PPSO), and Hybrid Grey Wolf Optimization (GWO). The core analysis centers on evaluating the performance of these integrated models based on key accuracy metrics and runtime efficiency. Notably, the results underscore that the XGBoost-SSA model emerges as the superior performer, exhibiting heightened accuracy and superior performance in predicting electricity consumption. This model showcases the highest coefficient of determination (R2) value and demonstrates lower errors during the testing phase, thereby presenting a promising and effective approach for electricity consumption prediction in the specific context of Turkey.

Comparative Analysis of Machine Learning Techniques in Air Quality Index (AQI) prediction in smart cities

Comparative Analysis of Machine Learning Techniques in Air Quality Index (AQI) prediction in smart cities

Analysis and Comparison of Machine Learning Techniques for DDoS Attack Classification in Network Environments

This research presents a comparative analysis of machine learning techniques for classifying Distributed Denial of Service (DDoS) attacks within network traffic. We evaluated the performance of three algorithms: Logistic Regression, Decision Tree, and Random Forest, including their scaled-feature counterparts. The study utilized a robust methodology incorporating advanced data preprocessing, feature engineering, and Synthetic Minority Over-sampling Technique (SMOTE) to address class imbalance. The models were rigorously tested using a cross-validation framework, assessing their accuracy, precision, recall, and F1 score. Results indicated that the Random Forest algorithm outperformed the others, demonstrating superior predictive accuracy and consistency, albeit with higher computational costs. Logistic Regression, when feature-scaled, showed significant improvement in performance, highlighting the importance of data normalization in models sensitive to feature scaling. Decision Trees provided a quick and interpretable model, though slightly less accurate than the Random Forest. The research findings highlight the trade-offs between predictive performance and computational efficiency in selecting machine learning models for cybersecurity applications. The study contributes to the cybersecurity domain by elucidating the efficacy of ensemble techniques in DDoS attack classification and underscores the potential for model improvement through scaling and data balancing.

A comparative study of ensemble learning algorithms for high-frequency trading

High-Frequency Trading utilizes powerful mathematical algorithms to execute transactions at an extremely rapid pace, which makes the use of machine learning techniques for prediction necessary. This paper evaluates the effectiveness of various ensemble learning algorithms, including Boosting (Adaboost and XGBoost), Bagging (Random Forest and Bagging-LSVM), and Stacking, in predicting stock prices using High-Frequency Trading (HFT) data from the Casablanca Stock Exchange. The data contains 311.812 transactions at the millisecond precision level and was obtained under a non-disclosure agreement. Three performance metrics are used: Root Mean Squared Error (RMSE), Mean Absolute Error (MAE) and Mean of the Squared Errors (MSE) and accuracy of predictions is assessed daily, monthly and annually. The study shows that the Stacking Model outperforms other algorithms in forecasting prices across different periods due to its ability to generate profit from multiple learners and construct a robust model. These results provide valuable insights for improving High-Frequency Trading (HFT) strategies, including the accuracy and robustness of the algorithms in predicting stock prices, which are crucial for effective HFT strategies. The findings highlight how each algorithm characteristics and performance metrics can enhance decision-making in HFT. The novelty of this study lies in its unique combination of a focus on an emerging market, the application of advanced ensemble learning algorithms to high-frequency data, and its comprehensive approach to evaluating algorithm performance. Also, this paper is the first study that focuses on applying ensemble learning algorithms to high-frequency data in Casablanca Stock Exchange and one of the rare that focuses on African stock markets. Thus, this study is pioneering in its comparative analysis of machine learning techniques for HFT in Morocco, introducing sophisticated and more accurate tools for analyzing and predicting Casablanca Stock Exchange trends.

Time Period Categorization in Fiction: A Comparative Analysis of Machine Learning Techniques

This study investigates the automatic categorization of time period metadata in fiction, a critical but often overlooked aspect of cataloging. Using a comparative analysis approach, the performance of three machine learning techniques, namely Latent Dirichlet Allocation (LDA), Sentence-BERT (SBERT), and Term Frequency-Inverse Document Frequency (TF-IDF) were assessed, by examining their precision, recall, F1 scores, and confusion matrix results. LDA identifies underlying topics within the text, TF-IDF measures word importance, and SBERT measures sentence semantic similarity. Based on F1-score analysis and confusion matrix outcomes, TF-IDF and LDA effectively categorize text data by time period, while SBERT performed poorly across all time period categories.

Comparative analysis of machine learning techniques for cryptocurrency price prediction

The emergence of cryptocurrencies has revolutionized the concept of digital currencies and attracted significant attention from financial markets. Predicting the price dynamics of cryptocurrencies is crucial but challenging due to their highly volatile and non-linear nature. This study compares the performance of various models in predicting cryptocurrency prices using three datasets: Bitcoin (BTC), Litecoin (LTC), and Ethereum (ETH). The models analyzed include Moving Average (MA), Logistic Regression (LR), Autoregressive Integrated Moving Average (ARIMA), Long Short-Term Memory (LSTM), and Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM). The objective is to uncover underlying patterns in cryptocurrency price movements and identify the most accurate and reliable approach for predicting future prices. Through the analysis, it could be observed that MA, LR, and ARIMA models struggle to capture the actual trend accurately. In contrast, LSTM and CNN-LSTM models demonstrate strong fit to the actual price trend, with CNN-LSTM exhibiting a higher level of granularity in its predictions. Results suggest that deep learning architectures, particularly CNN-LSTM, show promise in capturing the complex dynamics of cryptocurrency prices. These findings contribute to the development of improved methodologies for cryptocurrency price prediction.

A comparative analysis of machine learning techniques for aboveground biomass estimation: A case study of the Western Ghats, India

Accurate assessment of aboveground biomass (AGB) in tropical forests, particularly within a biodiversity hotspot, is vital for sustainable resource management and the preservation of ecosystems. However, estimating AGB in tropical forests is complex due to the diverse and intricate nature of vegetation, necessitating the integration of data from multiple sources. To tackle this challenge, our study utilized seven machine learning algorithms to analyze various combination of multisource datasets. We developed seven models/scenarios that incorporated Sentinel-1, Sentinel-2 as well as environmental factors such as topography, soil and climate to identify key variables for accurate estimation of AGB. For optimal performance, hyperparameters of the algorithms were fine-tuned through 10-fold cross-validation and their accuracy were assessed using the testing dataset. We found that the integrated model of satellite datasets, topography, climate, and soil variables exhibited the highest accuracy, where ensemble stacking, that combined multiple MLAs, proved to be reliable and best suited for predicting AGB (mean absolute error-3.97 Mg 0.1 ha−1, root mean square error-5.67 Mg 0.1 ha−1, and coefficient of determination - 0.82). Notably, the top predictor variables included Sentinel-2 bands (near infrared and green), soil properties (pH and soil organic carbon), and topography (elevation). The study emphasizes the significance of incorporating environmental variables (specifically topography and soil properties) along with Sentinel datasets to improve the accuracy of AGB estimation. This approach has the potential for broader applications, specifically in regions where vegetation productivity is governed by diverse environmental conditions.

Mathematical Modeling and Statistical Analysis of Elederly Fall Detection System Using Improved Support Vector Machine

This research focuses on enhancing the safety of elderly individuals through early fall detection using mathematical modelling and statistical analysis of machine learning techniques for the application and effectiveness of elderly fall detection. Falls among the elderly can lead to severe consequences, necessitating timely intervention. Leveraging machine learning algorithms, this innovative open-source project analyses sensor data from wearable sensors like accelerometers, gyroscopes, and magnetometers, along with environmental data such as temperature and humidity, to promptly identify fall patterns. The project uses a dataset containing 14 variables, including age, sex, medical indicators, and more, collected from diverse subjects and activities. The results obtained during the testing phase underscore the importance of refining the model through dataset adjustments. As the physical, cognitive, and sensory functions decline with age, the risk of falls increases, highlighting the need for fall detection and prevention systems. This research reviews the latest machine learning-based systems for fall detection and prevention, analyzing them based on various parameters. It identifies support vector machines and wearable devices as common tools, but emphasizes the need for broader studies in different contexts. The paper also visualizes the performance metrics of ML algorithms in conjunction with various wearables and outlines future research directions, including energy efficiency, sensor fusion, context awareness, and wearable design, to advance fall detection and prevention for the elderly.

Enhancing Predictive Quality in HVOF Coating Technology: A Comparative Analysis of Machine Learning Techniques

This paper presents a comparative analysis of the performance of machine learning techniques for enhancing predictive quality in thermal spray coating technology, with a focus on the high velocity oxygen fuel (HVOF) process. The study aims for accurately predicting coating properties, which are pivotal in determining coating quality in HVOF applications. By investigating five influential factors, including the fuel-to-oxygen ratio and coating velocity, this research was designed to improve the understanding and control of these properties. Through advanced statistical design of experiments, correlations between these factors and key coating properties are established. Six machine learning models are trained and evaluated to assess their predictive capabilities for coating properties. The obtained results provide valuable insights into the HVOF coating process and highlight the potential of predictive model-based techniques for accurately predicting such properties. Furthermore, this research offers guidance for selecting an effective machine learning technique in the context of thermal spray technology.

A Performance Analysis of Machine Learning Techniques for Credit Card Fraud Detection

A Performance Analysis of Machine Learning Techniques for Credit Card Fraud Detection

Telecom customer churn prediction model : Analysis of machine learning techniques for churn prediction and factor identification in telecom sector

This research project leverages exploratory data analysis (EDA) on a telecom company’s customer data to predict user churn. Utilizing Python and its libraries, including Pandas, NumPy, Matplotlib, and Scikit-Learn, it identifies key parameters crucial for accurate predictions. The results and predictions are visualized using the Flask framework and Power BI analytics tool. In the highly competitive telecommunications sector, customer churn poses a significant challenge, with an annual churn rate of 15-25%. This study addresses the escalating phenomenon of consumers freely switching between providers, leading to financial losses for businesses. By employing machine learning models, it aims to forecast potential churners, enabling companies to focus targeted retention efforts and mitigate losses. The tool’s efficacy lies in its ability to analyze patterns among churned users, offering a solution to a pressing issue in the telecom industry. Beyond its immediate application, this tool can be extended to other industries, providing valuable insights for customer retention strategies.

A Survey on Information Bottleneck.

This survey is for the remembrance of one of the creators of the information bottleneck theory, Prof. Naftali Tishby, passing away at the age of 68 on August, 2021. Information bottleneck (IB), a novel information theoretic approach for pattern analysis and representation learning, has gained widespread popularity since its birth in 1999. It provides an elegant balance between data compression and information preservation, and improves its prediction or representation ability accordingly. This survey summarizes both the theoretical progress and practical applications on IB over the past 20-plus years, where its basic theory, optimization, extensive models and task-oriented algorithms are systematically explored. Existing IB methods are roughly divided into two parts: traditional and deep IB, where the former contains the IBs optimized by traditional machine learning analysis techniques without involving any neural networks, and the latter includes the IBs involving the interpretation, optimization and improvement of deep neural works (DNNs). Specifically, based on the technique taxonomy, traditional IBs are further classified into three categories: Basic, Informative and Propagating IB; While the deep IBs, based on the taxonomy of problem settings, contain Debate: Understanding DNNs with IB, Optimizing DNNs Using IB, and DNN-based IB methods. Furthermore, some potential issues deserving future research are discussed. This survey attempts to draw a more complete picture of IB, from which the subsequent studies can benefit.

A Comparative Analysis of Machine Learning Techniques for National Glacier Mapping: Evaluating Performance through Spatial Cross-Validation in Perú

Accurate glacier mapping is crucial for assessing future water security in Andean ecosystems. Traditional accuracy assessment may be biased due to overlooking spatial autocorrelation during map validation. In recent years, spatial cross-validation (CV) strategies have been proposed in environmental and ecological modeling to reduce bias in predictive accuracy. In this study, we demonstrate the influence of spatial autocorrelation on the accuracy assessment of glacier surface predictive models. This is achieved by comparing the performance of several widely used machine learning algorithms including the gradient-boosting machines (GBM), k-nearest neighbors (KNN), random forest (RF), and logistic regression (LR) for mapping nine main Peruvian glacier regions. Spatial and non-spatial cross-validation methods were used to evaluate the model’s classification errors in terms of the Matthews correlation coefficient. Performance differences of up to 18% were found between bias-reduced (spatial) and overoptimistic (non-spatial) cross-validation results. Regarding only spatial CV, the k-nearest neighbors were the overall best model across Huallanca (0.90), Huayhuasha (0.78), Huaytapallana (0.96), Raura (0.93), Urubamba (0.96), Vilcabamba (0.93), and Vilcanota (0.92) regions, consistently demonstrating the highest performance followed by logistic regression at Blanca (0.95) and Central (0.97) regions. Our validation approach, accounting for spatial characteristics, provides valuable insights for glacier mapping studies and future efforts on glacier retreat monitoring. Incorporating this approach improves the reliability of glacier mapping, guiding future national-level initiatives.

Towards Sustainable Equine Welfare: Comparative Analysis of Machine Learning Techniques in Predicting Horse Survival

Promoting sustainable equine welfare is pivotal in ensuring the well-being of horses, particularly concerning their survival based on past medical conditions. This study presents a comprehensive comparative analysis of various machine learning techniques employed to predict the survival prospects of horses using historical medical data. By leveraging a dataset encompassing diverse medical attributes and survival outcomes, this research assesses the efficacy and comparative performance of distinct machine learning algorithms. The study delves into the application of supervised learning models, including but not limited to decision trees, random forests, support vector machines, and neural networks, in predicting equine survival. Evaluative metrics such as accuracy, precision, recall, and F1 score are employed to assess the predictive capabilities and generalizability of each model. Moreover, this research emphasizes the importance of sustainable equine welfare within the broader context of responsible animal care.

Machine Learning Techniques for Vertical Lidar-Based Detection, Characterization, and Classification of Aerosols and Clouds: A Comprehensive Survey

This survey presents an in-depth analysis of machine learning techniques applied to lidar observations for the detection of aerosol and cloud optical, geometrical, and microphysical properties. Lidar technology, with its ability to probe the atmosphere at very high spatial and temporal resolution and measure backscattered signals, has become an invaluable tool for studying these atmospheric components. However, the complexity and diversity of lidar technology requires advanced data processing and analysis methods, where machine learning has emerged as a powerful approach. This survey focuses on the application of various machine learning techniques, including supervised and unsupervised learning algorithms and deep learning models, to extract meaningful information from lidar observations. These techniques enable the detection, classification, and characterization of aerosols and clouds by leveraging the rich features contained in lidar signals. In this article, an overview of the different machine learning architectures and algorithms employed in the field is provided, highlighting their strengths, limitations, and potential applications. Additionally, this survey examines the impact of machine learning techniques on improving the accuracy, efficiency, and robustness of aerosol and cloud real-time detection from lidar observations. By synthesizing the existing literature and providing critical insights, this survey serves as a valuable resource for researchers, practitioners, and students interested in the application of machine learning techniques to lidar technology. It not only summarizes current state-of-the-art methods but also identifies emerging trends, open challenges, and future research directions, with the aim of fostering advancements in this rapidly evolving field.

A Comparative Analysis of Machine Learning Techniques for Cyberbullying Detection on FormSpring in Textual Modality

Social media usage has increased tremendously with the rise of the internet and it has evolved into the most powerful networking platform of the twenty-first century. However, a number of undesirable phenomena are associated with increased use of social networking, such as cyberbullying (CB), cybercrime, online abuse and online trolling. Especially for children and women, cyberbullying can have severe psychological and physical effects, even leading to self-harm or suicide. Because of its significant detrimental social impact, the detection of CB text or messages on social media has attracted more research work. To mitigate CB, we have proposed an automated cyberbullying detection model that detects and classifies cyberbullying content as either bullying or non-bullying (binary classification model), creating a more secure social media experience. The proposed model uses Natural Language Processing (NLP) techniques and Machine Learning (ML) approaches to assess cyberbullying contents. Our main goal is to assess different machine learning algorithms for their performance in cyberbullying detection based on a labelled dataset from Formspring [1]. Nine popular machine learning classifiers namely Bootstrap Aggregation or Bagging, Stochastic Gradient Descent (SGD), Random Forest (RF), Decision Tree (DT), Linear Support Vector Classifier (Linear SVC), Logistic Regression (LR), Adaptive Boosting (AdaBoost), Multinomial Naive Bayes (MNB) and K-Nearest Neighbour (KNN) are considered for the work. In addition, we have experimented with a feature extraction method namely CountVectorizer to obtain features that aid for better classification. The results show that the classification accuracy of AdaBoost classifier is 86.52% which is found better than all other machine learning algorithms used in this study. The proposed work demonstrates the effectiveness of machine learning algorithms in automatic cyberbullying detection as against the very intense and time-consuming approaches for the same problem, thereby by facilitating easy incorporation of an effective approach as tools across different platforms enabling people to use social media safely.

Comparative analysis of machine learning techniques for so2 prediction modelling

Sulphur dioxide (SO2) is produced both naturally and by human activity. The primary natural resource is derived from volcanoes. The burning of fossil fuels is the primary anthropogenic source (especially coal and diesel). Therefore, a reliable and accurate predicting method is essential for an early warning system for SO2 atmospheric concentration. There are still limited studies in Malaysia that use machine learning methods to predict SO2 concentrations. With the aid of machine learning, this study seeks to develop and predict future SO2 concentrations for the next day using the maximum daily data from Klang, Selangor. RapidMiner Studio is the data mining tool used for this research work. Based on the results, it showed that the SVM model was the best guide to be used compared with the other five models (GLM, DL, DT, GBT, and RF). The performance indicators showed that the SVM model was adequate for the next day’s prediction (R2 = 0.77, SE = 8.26, REL = 18.69%, AE = 1.46, and RMSE = 2.82). The developed model in this research can be used by Malaysian authorities as a public health protection measure to give Malaysians an early warning about the problem of air pollution. The goal of predictive modelling is to make a reasonable prediction of the variable of interest, and frequently, to determine how much the independent variable contributed to the dependent variable. The results also showed that the previous SO2 concentrations were one of the most influential parameters used to predict the future SO2 concentrations.